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Abstract 

A  fast-growing  trend  in  e-leaming  environments  is  the  investment  in  avatar 
technology  to  deliver  an  engaging  and  interesting  learning  experience.  E-leaming  itself 
has  limited  and  inconsistent  research  into  its  learning  effectiveness,  and  this  is  especially 
true  for  innovative  avatar  instructional  methods.  The  purpose  of  this  research  was  to 
develop  a  generalizable  theory  that  can  be  used  to  assess  the  learning  effectiveness  of 
various  media  types  used  in  e-learning  environments.  A  research  model  was  developed 
and  used  to  study  a  U.S.  Air  Force  Squadron  Officer  School  Distance  Learning  program 
that  used  avatar,  video,  audio,  and  text-based  scenarios  to  reinforce  learning  objectives. 

It  was  hypothesized  that  media  with  higher  levels  of  learning  engagement  would  lead  to 
more  favorable  reactions  and  thus  higher  levels  of  understanding.  While  text  and  audio 
showed  positive  learning  engagement  results,  the  hypotheses  for  both  avatar  and  video 
influence  on  learning  engagement  were  not  supported.  Results  also  showed  full  and 
partial  support  for  learning  engagement  leading  to  favorable  learning  outcomes.  The 
model  developed  in  this  research  identified  the  learning  effectiveness  of  an  e-leaming 
program  and  can  be  used  to  guide  education  and  training  investment  decisions  based  on 
proven  learning  outcomes  rather  than  the  surface  appeal  of  emotional  interest  and 
engagement  features. 
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AVATARS,  MEDIA  USAGE,  AND  THE  LINKAGES  TO 
E-LEARNING  EFFECTIVENESS 


I.  Introduction 

“I  have  no  special  talents.  lam  only  passionately  curious.  ” 

-  Albert  Einstein 

E-leaming  is  defined  as  “training  delivered  on  a  computer  that  is  designed  to 
support  individual  learning  or  organizational  perfonnance  goals”  (Clark  and  Mayer, 
2008).  As  e-leaming  continues  to  grow  in  popularity,  the  use  of  avatars  is  the  latest 
innovative  trend  in  organizational  education  and  training  environments.  An  avatar  has 
been  described  as  an  animated  character  who  guides  learners  through  a  course  of  e- 
leaming  (Fee,  2009).  The  most  noticeable  impact  avatars  are  having  on  learning  is  the 
widespread  adoption  of  the  technology  along  with  a  great  deal  of  trust  placed  in  its  ability 
to  produce  required  learning  results.  An  American  Society  for  Training  and 
Development  study  showed  an  estimated  $125.88  billion  was  spent  by  U.S.  organizations 
in  2009  on  employee  learning  and  development,  with  36%  of  that  spent  on  e-leaming 
(Patel,  2010).  Unfortunately,  as  substantial  investments  on  avatar  innovations  and  other 
e-leaming  methods  take  place,  there  is  very  little  research  to  support  learning  outcomes. 

IT-Performance  Paradox 

Organizations  seeking  to  maximize  value  and  performance  by  adopting  e-learning 
systems  may  be  falling  victim  to  the  IT-performance  paradox  as  described  in  a  2008 
perspective  of  the  prisoner ’s  dilemma  (Tangpong,  2008).  The  IT-performance  paradox 
suggests  that  while  information  technology  (IT)  innovations  promise  perfonnance 
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improvements,  often  it  is  only  the  vendors  who  realize  any  gains.  The  marketing  of  e- 
leaming  systems  has  made  the  technology  a  ubiquitous  commodity  simply  by  targeting 
organizations  with  education  and  training  needs.  In  a  rush  to  gain  performance 
improvements  while  satisfying  budget  efficiencies,  a  “bandwagon”  effect  has  occurred  as 
organizations  jump  on  board  with  the  latest  innovative  learning  systems  even  when  there 
has  been  no  empirical  evidence  to  show  significant  improvement  in  performance  or 
efficiencies.  The  result  of  such  a  paradox,  as  Tangpong  observes,  is  there  may  be  no  real 
value  created  for  investors,  leaving  only  the  vendor(s)  with  any  substantial  value/payoff. 
The  paradox  can  also  be  attributed  to  what  Bates  and  Poole  describe  as  a  technological 
imperative :  “One  frequent  criticism  of  the  use  of  technology  for  teaching  in  academic 
circles  is  that  we  are  being  driven  by  a  technological  imperative:  we  have  to  use 
technology  because  of  a  blind  belief  that  it  is  good  for  us.  If  we  don’t  agree  to  the  use  of 
technology,  we  will  be  considered  out  of  date  and  may  lose  our  credibility”  (2003). 

Total  Cost  of  Ownership 

Aside  from  the  initial  e-learning  software  purchase(s),  there  are  typically  other 
costs  involved,  such  as  the  subsequent  hardware  upgrades  that  accompany  e-learning 
investments.  For  example,  Sheldon  Parks,  the  gaming  integrator  for  the  U.S.  Army 
Training  and  Doctrine  Command’s  Capability  Manager  for  Gaming,  said  that  in  a  move 
to  “make  every  piece  of  hardware  in  the  battle  command  centers  gaming-capable,”  the 
Army  is  upgrading  the  video  cards  and  microprocessors  in  computers  at  all  Battle 
Command  Training  Centers  (Peck,  2011).  Although  important,  this  initial 
implementation  commitment  and  the  subsequent  total  cost  of  ownership  (TCO)  factors 
(technical  support,  training,  maintenance,  lifecycle  replacements,  and  so  on),  should  be 
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secondary  to  the  question  of  whether  or  not  the  learning  technology  is  effective  to  begin 
with.  This  study  posits  that  there  is  no  justification  for  the  amount  of  spending  on  e- 
leaming  taking  place  today  without  empirical  evidence  of  the  method’s  learning 
effectiveness. 

The  Unstoppable  Force 

Research  into  the  e-learning  trend  is  limited,  inconsistent,  and  inconclusive 
(Bostrom  and  Gupta,  2009).  Current  studies  reaffirm  past  findings  that  show  there  are  no 
significant  differences  in  e-learning  outcomes  as  compared  with  face-to-face  instruction 
(Carrol  and  Burke,  2010),  and  continue  to  declare  a  need  for  more  research  into  e- 
leaming  effectiveness  (Mayer,  2005;  Gupta  and  Bostrom,  2010).  Furthermore,  the  TCO 
and  short  lifecycles  of  modern  technology  may  be  prohibiting  overall  cost-savings  while 
continually  presenting  researchers  a  moving  target  that  hinders  the  establishment  of  any 
meaningful  learning  theories  that  can  shed  light  on  the  effectiveness  of  modern  e-learning 
instructional  system  designs.  However,  e-learning  has  been  considered  an  “unstoppable 
force”  (Spencer,  1999),  and  if  that  is  so,  it  is  imperative  that  the  research  and  education 
community  collectively  confront  the  task  of  determining  how  to  maximize  learning 
outcomes  in  our  e-learning  environments. 

The  e-Learning  Marketplace 

The  continuous  and  rapid  growth  of  technology  has  allowed  the  development  and 
implementation  of  a  flood  of  multimedia  e-leaming  environments,  many  of  which  are 
implemented  with  the  assistance  of  accommodating  vendors  who  tend  to  place  a  heavy 
emphasis  on  the  technological  characteristics  of  instructional  system  designs,  and  little  on 
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the  learning  content.  Fee  describes  why  he  feels  there  is  such  a  technology  focus  in  the  e- 
leaming  marketplace: 

There  are  three  component  parts  of  e-learning,  namely  enabling 
technology,  learning  content  and  learning  design.  People  tend  to  focus  on 
the  first,  the  technology,  because  this  is  the  new  and  unfamiliar 
component,  but  the  other  two  are  at  least  as  important.  Software  vendors 
tend  to  place  great  emphasis  on  the  first,  because  that  is  what  they 
contribute,  and  where  they  make  their  living,  but  that  does  not  mean  that 
learning  and  development  professionals  need  to  follow  suit  (p.  16). 

Scientific  Approach 

As  the  emotional  appeal  of  innovative  learning  technologies  used  in  e-learning 
continues  to  grip  the  education  and  training  landscape,  attention  needs  to  be  turned  more 
toward  the  fundamentals  and  outcomes  of  learning.  The  multimedia  principle  states  that: 
“People  can  learn  more  deeply  from  words  and  pictures  than  from  words  alone”  (Mayer, 
2005,  p.  15).  This  is  the  basis  for  most  multimedia  learning  research  when  considering 
the  differences  in  delivery  methods. 

If  Mayer’s  multimedia  principle  holds  true,  the  theories  and  observations 
presented  in  this  study  provide  promise  and  support  for  the  effective  use  of  avatars  in  e- 
leaming  environments.  This  paper  suggests  a  prudent  approach  to  e-learning  by  ensuring 
that  cognitive  and  learning  sciences  are  given  at  least  the  same  attention  as  the  innovative 
technologies  used  during  the  development,  acquisition,  and  implementation  of 
instructional  system  designs  (DeRouin  et  ah,  2005). 

The  Avatar  Advantage? 

The  advent  of  high-speed  internet  availability  has  enabled  multimedia  delivery 
capabilities  never  seen  before  on  such  a  ubiquitous  scale.  With  its  innovative  virtual 
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moving  pictures,  audio  features,  interactivity  and  emotional  appeal,  will  avatars  used  in 
e-leaming  epitomize  the  multimedia  principle?  It  may  be  they  turn  out  to  be  a  truly 
revolutionary  method  for  increasing  learning  effectiveness  due  to  their  ability  to  engage  a 
learner  in  an  e-leaming  experience.  To  date,  there  has  been  no  conclusive  evidence  to 
show  the  use  of  avatars  lead  to  favorable  learning  outcomes  over  other  media  channels 
and  can  therefore  can  only  be  viewed  as  a  speculative  instructional  technique  at  this 
point.  This  study  seeks  to  discover  how  the  use  of  avatars  and  other  media  channels 
affect  learning  engagement  and  discover  if  avatars  do  indeed  hold  a  definitive  learning 
advantage  over  other  media  channels  of  instruction. 

Research  Objectives 

The  purpose  of  this  research  is  to  gain  insight  into  the  impact  media  usage  has  on 
an  e-learning  environment  by  determining  which  media  channels  influence  favorable 
learning  outcomes  most  effectively.  This  study  is  an  attempt  to  turn  an  overwhelming 
focus  away  from  the  “technological  imperative”  mindset  and  toward  a  “learning 
imperative”  that  accentuates  the  use  of  cognitive  and  learning  sciences.  To  do  this,  the 
study  examines  how  different  media  types  (1)  induce  learning  engagement,  (2)  invoke 
learner  reactions,  and  (3)  influence  learning. 

Implications 

This  study  fills  a  research  gap  in  the  area  of  using  e-learning  to  increase  the 
effectiveness  and  efficiency  of  learning  programs  (Bostrom  and  Gupta,  2009),  provides 
guidance  for  making  infonned  decisions  about  educational  investments,  and  will  benefit 
future  research  by  exploring  methods  for  improving  e-leaming  systems. 
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Problems  to  be  Investigated 

Emerging  technologies  continue  to  offer  new  ways  of  approaching  the  education 
and  training  needs  of  organizations.  Many  times  these  innovative  instructional  methods 
gain  widespread  acceptance  while  still  in  their  developmental  infancy  and  without  any 
empirically  validated  research  to  demonstrate  their  affects  on  learning  outcomes.  Avatar 
technology  is  an  example  of  one  such  innovation.  This  study  has  introduced  the  avatar 
channel  into  the  research  stream  with  the  primary  goal  of  developing  an  increased 
understanding  of  how  e-learning  media  channel  options  influence  learning  outcomes. 

To  achieve  this  goal  and  help  fill  a  gap  in  existing  e-learning  research,  the 
following  research  question  was  developed:  To  what  extent  does  the  use  of  avatar,  video, 
audio,  and  text-based  instructional  technologies  impact  learning  outcomes  in  an  e- 
learning  environment?  In  order  to  gain  the  necessary  insight  to  answer  the  overarching 
research  question,  this  thesis  will  focus  on  the  following  three  core  investigative 
questions: 

(1)  To  what  extent  does  media  usage  in  e-learning  environments  affect  learning 
engagement? 

(2)  To  what  extent  does  learning  engagement  affect  learning  affective  and 
cognitive  reactions? 

(3)  To  what  extent  do  learner  affective  and  cognitive  reactions  affect  subjective 
learning  outcomes? 
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Methodology 

This  study  used  a  quantitative  research  methodology  and  was  accomplished  through 
the  analysis  of  data  gathered  from  a  survey  instrument  using  Likert-scale  items.  These 
items  were  developed  to  measure  learning  engagement,  affective  and  cognitive  reactions, 
and  subjective  learning. 

Thesis  Overview 

The  topics  discussed  in  this  chapter  included  the  background,  purpose  and 
significance  of  the  study,  problem  to  be  investigated,  and  research  methodology.  The 
next  chapter  (chapter  two)  provides  an  extensive  review  of  relevant  theories/models 
related  to  infonnation  technology  and  educational  psychology,  specifically  addressing 
media  usage  linkages  with  learning  engagement  and  learning  outcomes.  Based  on  the 
literature  review,  hypotheses  were  developed  pertaining  to  how  the  use  of  different  media 
channels  would  influence  learning  engagement,  and  lead  to  certain  learning  outcomes. 
Chapter  three  then  discusses  the  research  design,  sample  characteristics,  procedure,  and 
measures  that  formulate  the  methodology  used  in  this  study.  Next,  chapter  four  examines 
the  quantitative  data  acquired  from  the  data  collect  with  emphasis  on  the  statistical 
significance  of  the  findings  based  on  the  hypotheses  proposed  in  this  study.  Finally, 
chapter  five  provides  a  summary  of  the  findings,  limitations,  theoretical  implications, 
applications,  and  opportunities  for  future  research. 
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II.  Literature  Review 


Overview 

The  primary  purpose  of  this  literature  review  is  to  assess  the  current  state  of 
knowledge  with  respect  to  the  influencing  elements  of  media  channels  in  hopes  of 
understanding  how  and  why  people  learn  most  effectively  in  e-learning  environments. 
This  review  lays  the  groundwork  required  to  help  fill  a  research  gap  that  exists  in  the  area 
of  cognitive  learning  processes  in  organizational-based  settings  (Bostrom  and  Gupta, 
2009),  and  is  structured  as  follows. 

First,  a  background  of  the  literature  is  presented.  Second,  a  description  of  the 
theoretical  foundations  applied  to  this  study.  Then,  the  media  channels  examined  in  this 
study  are  discussed.  Finally,  the  links  between  media  usage,  learning  engagement,  and 
learning  outcomes  are  presented  along  with  the  hypotheses  developed. 

Background 

E-leaming  has  revolutionized  education  and  training  by  introducing  innovative 
methods  of  instruction  and  breaking  the  traditional  constraints  of  time  and  location 
through  the  enabling  global  reach  of  the  Internet  (Brown  and  Adler,  2008).  However,  a 
cautious  approach  to  such  innovations  has  been  advised  in  the  wake  of  the  technological 
imperative  mentioned  earlier  that  states  people  are  persuaded  to  implement  technological 
solutions  for  fear  of  losing  credibility  and  do  so  with  a  blind  believe  that  technology  is 
somehow  good  for  them. 

In  order  to  most  effectively  develop  and  use  e-learning  systems,  a  greater 
understanding  of  how  people  learn  in  these  environments  is  needed.  With  a  focus  on  the 


area  of  media  usage  and  its  impact  in  e-leaming,  this  section  examines  relevant  literature 
associated  with  learning  engagement,  information  technology  theory,  and  learning 
outcomes  to  help  reveal  optimal  learning  conditions  in  e-leaming. 

Theoretical  Foundation 

The  theoretical  underpinnings  of  this  study  are  based  on  research  into  information 
technology  and  educational  psychology,  specifically  learning  engagement  and  learning 
outcomes.  This  theoretical  approach  was  adopted  from  the  concepts  built  into  the  Alavi 
and  Leidner  (2001)  framework  for  e-learning  research  (Figure  1).  For  the  purposes  of 
this  research,  the  information  technology  portion  of  this  framework  refers  to  the  enabling 
capabilities  that  allow  the  delivery  of  the  four  media  channels  used  in  this  study. 
Consistent  with  the  framework,  it  is  theorized  here  that  media  usage  will  initiate  various 
levels  of  learning  engagement,  thereby  triggering  the  psychological  learning  processes 
that  ultimately  detennine  learning  effectiveness. 


Figure  1.  A  framework  for  e-learning  research  (Alavi  and  Leidner,  2001) 
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Media  Usage 

Four  optional  media  channels  were  examined  in  this  study  to  determine  how  their 
usage  affected  learning  effectiveness:  text,  audio,  video,  and  avatar.  Some  media 
required  the  user  to  be  actively  connected  to  the  e-learning  program  for  content  access, 
whereas  others  allowed  printing  and  downloadable  access. 

The  media  channels  were  instructional  methods  a  user  could  choose  to  engage  in 
to  reinforce  their  understanding  of  course  objectives.  Each  channel  was  used 
independent  of  the  other;  however,  some  may  have  contained  an  element  of  the  other. 

For  example,  video  and  avatar  also  include  audio  channels,  however  those  elements  were 
considered  inherent  to  that  particular  instructional  method.  The  four  channels  are 
explained  briefly  as  follows: 


Text 

Coupled  with  mandatory  reading  assignments,  optional  text  scenarios  were 
presented  along  with  projects  to  be  completed  such  as  written  analysis  and  after-action 
report  presentations.  Users  could  view  the  text  either  through  HTML,  PDF  fonnat  and 
had  the  option  to  print  the  material  for  off-line  reading  at  a  location  of  their  choosing. 
There  was  no  immediate  feedback  provided  to  confirm  understanding  or  report 
perfonnance  results. 


Audio 

Provided  through  downloadable  MP3  files  that  could  be  listened  to  while 
following  along  with  course  modules,  or  taken  with  the  learner  and  listened  to  anywhere 


10 


as  a  “podcast.”  Once  again,  this  was  a  media  option  and  had  no  immediate  feedback  to 
ensure  understanding.  Myriad  influx 

Video 

Video  was  presented  as  prerecorded  video  lesson  material  and  meant  to  reinforce 
current  learning  objectives.  Users  had  to  be  located  at  a  computer  terminal  and  have 
internet  access.  There  were  no  additional  assignments  or  immediate  feedback  provided 
to  confirm  understanding. 

Avatar 

4)  Avatar.  Presented  as  vignettes  that  animated  short  (one  to  two  minute)  learning 
scenarios  designed  to  reinforce  current  learning  objectives.  A  multiple  choice  question 
and  answer  session  followed  immediately  after  each  scenario  and  an  explanation  of  right 
and  wrong  answers  was  provided. 

Learning  Engagement 

Learning  engagement  has  been  defined  as  “the  student’s  psychological  investment 
in  learning...  the  amount  of  time  they  spend,  the  intensity  of  their  concentration,  the 
enthusiasm  they  express,  and  the  degree  of  care  they  show”  (Newman,  1989). 
Engagement  has  been  determined  to  be  a  shortfall  in  DL  programs  (Salas  et  ah,  2005, 
cited  in  Johnson  et  ah,  2009)  and  DL  has  been  considered  “...not  necessarily  more 
motivating  and  can  mislead  students  into  investing  less  mental  effort  to  learn”  (Clark, 
2009).  However,  if  used  properly,  positive  results  have  been  demonstrated  using 
information  technology  to  induce  learning  engagement  (Chen,  et  ah,  2010).  As  an 
antecedent  to  learning  reactions,  research  has  shown  engagement  to  be  an  effective 
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enabler  of  desirable  learning  outcomes  by  establishing  interest  and  building  excitement  in 
the  learner  (Carini,  et  al.,  2006). 

Media  Usage  and  Learning  Engagement  Linkages 

Learning  effectiveness  will  depend  on  how  effective  the  communications  are 
between  the  educator  and  student  in  a  learning  environment.  Based  on  information 
technology  communication  and  cognitive  theories,  the  relationship  between  media 
channels  and  learning  engagement  needs  to  be  specified.  To  develop  the  hypotheses  of 
this  research,  the  media  richness  theory,  media  synchronicity  theory,  and  emotional 
interest  theory  were  used  to  predict  media  usage  and  learning  engagement. 

Media  Richness  Theory 

Daft  and  Lengel  described  media  richness  theory  as  the  “learning  capacity  of  a 
communication”  (1986).  Developed  to  help  identify  the  most  effective  media  for  a  given 
communication  task,  this  theory  suggests  avatars  have  an  advantage  over  the  other 
channels  due  to  the  immediate  feedback  provided  through  its  question  and  answer  feature 
and  its  ability  to  present  multiple  cues,  such  as  vocal  inflections  and  non-verbal  gestures. 
Additionally,  through  its  virtual  representation  of  people,  avatars  possess  a  “ social 
presence ,”  which  has  been  suggested  to  induce  motivation  when  coupled  with  a  rich 
media  (Dennis,  2005).  Considered  the  richest  media,  avatars  were  predicted  to  be  the 
most  engaging  and  result  in  the  most  usage. 

Media  Synchronicity  Theory 

Media  synchronicity  theory  (Dennis,  2008)  addresses  how  to  communicate  new 
infonnation  most  effectively  by  allowing  an  appropriate  amount  of  time  for  processing 
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the  new  information.  This  theory  terms  the  transmission  and  processing  of  new 
information  conveyance,  and  the  process  of  reaching  a  mutual  understanding  of  the 
information  as  convergence.  The  appropriate  speeds  at  which  conveyance  and 
convergence  occur  depend  on  the  complexity  of  the  information. 

With  the  intent  to  "generate  a  shared  understanding"  (2008,  p.  576),  the  act 
of  learning  is  considered  here  to  be  a  communication  process  between  teacher  (e-learning 
instructional  method)  and  student.  The  media  channels  used  in  this  study  are  examined 
to  determine  their  ability  to  convey  information  most  effectively  in  order  to  ensure  the 
learner  reaches  a  quick  understanding  of  course  material. 

The  use  of  audio,  text,  and  video  channels  were  often  lengthy  tasks  and  provided 
no  immediate  feedback.  Due  to  its  ability  to  facilitate  a  faster  understanding  through  its 
short  scenario-based  question  and  answer  feature,  avatars  were  again  predicted  to  engage 
learners  most  effectively  and  generate  the  desired  understanding. 

Emotional  Interest  Theory 

Emotional  interest  theory  states  that  the  use  of  seductive  details,  or  irrelevant  yet 
interesting  aspects,  can  energize  the  learner,  causing  them  to  pay  attention  and  learn  more 
as  a  result  (Harp  and  Mayer,  1998).  Text,  audio,  and  video  channels  are  predicted  to 
have  the  least  amount  of  these  engaging  qualities.  Modem  avatar  instructional  methods 
employ  attractive  graphical  and  audio  features  and  are  predicted  to  appeal  to  those 
interested  in  the  current  educational  innovations.  Therefore,  avatars  are  once  again 
thought  to  induce  the  most  learning  engagement  due  to  their  elements  of  appeal  as 
highlighted  by  the  emotional  interest  theory. 
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Hypotheses  la-d 

HI  (a):  Text  presentations  will  have  a  positive  relation  with  learning 
engagement. 

Hl(b):  Audio  presentation  will  be  more  engaging  than  text  and  account  for 
variance  in  learning  engagement  beyond  that  accounted  for  by  text 
presentation  alone. 

Hl(c):  Video  presentations  will  be  more  engaging  than  both  text  and  audio 
and  account  for  variance  in  learning  engagement  beyond  that  accounted  for 
by  text  and  audio  presentation. 

Hl(d):  Avatar  presentations  will  be  more  engaging  than  text,  audio,  and  video 
and  will  account  for  variance  in  learning  engagement  beyond  that  accounted 
for  by  text,  audio,  and  video  presentations  together. 


Learning  Outcomes:  Kirkpatrick’s  Levels  of  Evaluation 

In  1959,  Kirkpatrick  submitted  a  four-level  model  that  defines  techniques  for 
evaluating  training  programs.  The  first  two  levels,  reaction  and  understanding,  address 
the  training  environment  and  its  immediate  learning  outcomes.  The  second  two  levels, 
behavior  and  results,  address  behavioral  changes  and  organizational  results  caused  by  the 
training  and  are  outside  the  scope  of  this  study.  In  an  attempt  to  discover  the  impact  of 
media  usage  on  learning  outcomes,  this  study  applied  the  first  two  levels  of  the 
Kirkpatrick  model  for  measuring  learning  effectiveness. 

Level  1:  Reaction 

The  first  of  Kirkpatrick’s  levels  is  reaction.  Reactions  are  defined  as  how  trainees 
respond  to  a  particular  training  program  (Kirkpatrick,  1959).  Learner’s  reactions  in  this 
study  were  measured  both  affectively  and  cognitively.  Affective  reactions  are  considered 
to  be  a  learner’s  feelings  about  how  well  they  enjoyed  the  training  (i.e.,  fun/boring; 
pointless/useful).  Cognitive  reactions  indicate  how  receptive  the  learner  was  to  the 
learning  objectives  (i.e.,  easy  to  understand  or  confusing). 


14 


Learning  Engagement  and  Reactions 


Affective  reactions  were  hypothesized  in  this  study  to  have  a  positive  relationship 
with  learning  engagement:  the  more  the  learner  was  energized,  the  more  enjoyable  they 
will  perceive  the  experience  (Harp  and  Mayer,  1997).  Conversely,  cognitive  reactions 
were  predicted  to  have  a  negative  relation  with  learning  engagement. 

This  negative  relation  with  learning  engagement  is  consistent  with  Festinger’s 
theory  of  cognitive  dissonance  (1957),  and  is  considered  a  normal  part  of  the 
psychological  learning  process.  The  more  an  individual  attempts  to  leam,  the  more  they 
are  confronted  with  ideas  that  are  not  completely  congruent  with  their  own  ontological 
assumptions.  As  this  happens,  their  paradigms  and  psychological  schemas  are  challenged 
which  can  be  a  confusing  experience.  This  results  in  a  in  a  state  of  cognitive  dissonance 
(1957)  and  disequilibrium  that  can  be  uncomfortable. 

Hypotheses  2a-b 

H2(a):  Higher  levels  of  learning  engagement  will  lead  to  higher  levels  of 
favorable  affective  reactions 

H2(b)  higher  levels  of  learning  engagement  will  lead  to  a  negative  relation 
with  cognitive  reactions. 


Level  2:  Learning 

An  evaluation  of  learning  is  a  determination  of  knowledge  acquired,  skills 
improved,  or  attitudes  changed  due  to  the  training  (Kirkpatrick,  1996).  Only  subjective 
learning  measures  were  used  in  this  study  to  examine  a  learner’s  sense  of  increased 
knowledge,  improved  critical  thinking  skills,  heightened  cognitive  awareness,  and  so  on. 
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Kirkpatrick’s  original  model  was  updated  in  2006  to  include  a  “chain  of  evidence”  that 
accentuates  the  importance  of  connecting  each  successive  evaluation  level  to  the  next  for 
the  most  effective  learning  outcomes,  and  the  linkages  between  reactions  and  learning  are 
discussed  next. 


Reactions  and  Learning 

It  was  hypothesized  in  this  study  that  higher  levels  of  both  affective  and  cognitive 
reactions  would  lead  to  higher  levels  of  understanding.  The  emotional  interest  theory 
posits  that  higher  affective  reactions  (emotional  arousal)  indicate  enjoyment  with  the 
learning  experience  and  will  likely  lead  to  a  stronger  attention  span  thus  higher 
understanding  (Harp  and  Mayer,  1997).  Cognitively  speaking,  it  is  posited  here  that  as 
learners  faces  the  challenges  presented  by  cognitive  dissonance,  sensemaking  will 
eventually  occur  (Weick,  1979),  and  ultimately  result  in  positive  relations  with  the 
cognitive  reactions  that  lead  to  higher  levels  of  understanding. 


Hypotheses  3a-b 

H3(a):  Higher  levels  of  affective  reactions  will  lead  to  higher  levels  of 
understanding 

H3(b)  higher  levels  of  cognitive  reactions  will  lead  to  higher  levels  of 
understanding. 


Figure  2.  Research  model 
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III.  Methods 


Overview 

The  purpose  of  this  chapter  is  to  address  the  methodology  used  to  answer  the 
overarching  research  question  presented  in  this  study:  To  what  extent  does  the  use  of 
avatar,  video,  audio,  and  text-based  instructional  technologies  impact  learning  outcomes 
in  an  e-learning  environment?  This  chapter  examines  the  research  design  used  in  this 
study,  sample  population  characteristics,  location/setting,  procedures,  and  measures  used 
to  collect  the  data. 

Research  Design 

This  study  incorporated  a  quantitative  research  design  accomplished  using  data 
collected  from  a  21 -question  survey.  The  survey  was  comprised  of  5 -point  Likert  items 
that  measured  the  media  usage,  learner’s  engagement,  affective  and  cognitive  reactions, 
and  subjective  learning.  The  data  collected  using  the  applied  research  design  was  later 
analyzed  using  hierarchical  regression,  partial  correlations,  and  linear  regression  and  will 
be  covered  further  in  chapter  four. 

Sample  Characteristics 

The  sample  population  used  to  acquire  the  data  for  this  research  effort  consisted 
of  95  United  States  Air  Force  captains  (or  captain-selects)  and  other  eligible  DoD 
personnel  (Air  National  Guard,  Air  Force  reserve,  DoD  civilians,  Civil  Air  Patrol,  and 
other  US  services:  Anny,  Navy,  Marine  Corps,  and  Coast  Guard)  with  5-7  years  of 
service.  Thirty  percent  were  female  and  seventy  percent  were  male,  and  all  possessed  at 
least  a  bachelor’s  degree. 
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The  survey  participants  were  self-selected  and  had  completed  the  Squadron 
Officer  School  (SOS)  Distance  Learning  program,  a  military  Professional  Military 
Education  course  designed  to  teach  ethical  leadership,  decision  making,  and  team 
building  skills  to  developing  leaders. 

Procedure 

Air  Force  officers  were  self-enrolled  in  the  SOS  Distance  Learning  program  and 
were  given  up  to  eighteen  months  to  complete  the  required  course  material.  As  they 
progressed  through  course  modules,  they  were  presented  with  multimedia  options  (text, 
audio,  video,  and/or  avatar)  designed  to  reinforce  learning  objectives.  Upon  completion 
of  the  course,  volunteers  participated  in  an  anonymous  online  survey. 

This  survey  asked  for  participant’s  feedback  based  on  their  general  feelings  about 
the  learning  experience,  the  impact  of  the  optional  media  features,  and  perceived  levels  of 
subjective  learning.  The  survey  was  composed  of  single-item  usage  measures  and 
included:  media  type  used,  learning  engagement,  reactions  (Kirkpatrick  level  1),  and 
learning  (Kirkpatrick  level  2). 

Measures 

Usage :  This  was  a  5-point  Likert  scale  measurement  and  was  used  to  determine 
how  often  learners  used  the  optional  media  features.  The  scale  was  comprised  of  levels 
from  “almost  always”  to  “almost  never”  and  included  a  “not  applicable”  option. 

The  measures  for  media  usage  were  taken  using  a  5 -point  Likert  scale  with  an 
additional  “not  applicable”  option.  Specifically,  the  participants  were  asked: 
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1.  How  often  did  you  use  optional  TEXT  features  during  the  course? 

2.  How  often  did  you  use  optional  AUDIO  features  during  the  course? 

3.  How  often  did  you  use  optional  VIDEO  features  during  the  course? 

4.  How  often  did  you  use  optional  AVATAR  features  during  the  course? 

Engagement'.  This  scale  was  adopted  from  general  engagement  items  similar  to 
those  found  in  Kirkpatrick  Level  1  reaction  sheets  (2008)  and  guidelines  provided  by  the 
University  of  Minnesota  Center  for  Teaching  and  Learning  Services  (Ruhe,  2006)  . 
Example  items  from  this  scale  included  “I  devoted  my  full  attention  to  the  course,”  and 
“I  put  my  full  effort  into  completing  the  course  material.”  All  positively  worded  items 
were  retained  that  exhibited  low  cross  loading  when  subjected  to  an  exploratory  factor 
analysis.  An  eigenvalue  of  1.72  explained  89%  of  the  variance.  The  coefficient  alpha  for 
this  scale  was  .836. 

The  measures  for  learning  engagement  were  taken  using  a  5-point  Likert  scale. 
Specifically,  the  participants  were  asked  to  rate  the  following  statements: 

1.  I  devoted  my  full  attention  to  the  course. 

2.  I  stayed  focused  on  the  material  while  working  on  it. 

3.  I  put  my  full  effort  in  to  completing  the  course  material. 

Reaction :  This  scale  was  based  on  items  similar  to  those  found  on  Kirkpatrick 
Level  1  reaction  sheets  (2008).  A  single  exploratory  factor  analysis  yielded  a  two-factor 
solution.  The  first  factor  was  the  5  item  affective  reaction  scale  with  an  eigenvalue  of  3.3 
that  explained  41.4%  of  the  variance.  Example  items  from  this  scale  included,  “The 
course  was  interesting,”  and  “Aspects  of  the  course  were  confusing”  (reverse  coded). 

The  coefficient  alpha  for  this  scale  was  .808.  The  second  factor  was  a  3  item  cognitive 
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factor  scale  with  an  eigenvalue  of  1 .7  and  explained  2 1.5%  of  the  variance.  The 
coefficient  alpha  for  this  scale  was  .774. 

The  measures  for  reactions  were  taken  using  a  5-point  Likert  scale.  Specifically, 
the  participants  were  asked  to  rate  the  following  statements: 

1.  The  course  was  fun. 

2.  The  course  was  boring. 

3.  The  course  was  interesting. 

4.  The  course  seemed  poin  tless  to  me. 

5.  The  purpose  of  the  course  seemed  straightforward. 

6.  The  instructions  for  the  course  were  easy  to  understand. 

7.  The  course  material  did  not  make  sense  to  me. 

8.  Aspects  of  the  course  were  confusing. 

Understanding:  A  scale  was  created  that  examined  to  what  degree  a  student 
gained  an  understanding  of  the  learning  objectives.  When  the  complete  set  of  five 
original  items  were  subjected  to  a  principle  component  analysis  with  varimax  rotation, 
one  factor  emerged  based  on  the  Kaiser  Criterion  with  an  eigenvalue  of  4.84  that 
explained  80.6%  of  the  variance.  Examples  of  these  items  include,  “The  SOS  Distance 
Learning  course  has  helped  increase  my  critical  thinking  skills,”  and  “I  understand  how 
the  SOS  Distance  Learning  course  will  improve  my  performance.”  The  coefficient 
alpha  for  this  scale  was  .95. 

The  measures  for  subjective  learning  were  taken  using  a  5-point  Likert  scale. 
Specifically,  the  participants  were  asked  to  rate  the  following  statements: 
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1.  I  had  many  clarifying  moments  while  in  the  SOS  Distance  Learning  course. 

2.  The  material  covered  in  the  SOS  Distance  Learning  course  caused  me  to  think 
about  my  thinking. 

3.  The  SOS  Distance  Learning  course  has  helped  increase  my  critical  thinking 
skills. 

4.  I feel  the  SOS  Distance  Learning  course  has  been  of  value  to  me. 

5.  My  knowledge  and  skills  have  increased  since  taking  the  SOS  Distance 
Learning  course. 

6.  I  understand  how  the  SOS  Distance  Learning  course  will  improve  my 
performance. 
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IV.  Analysis  and  Results 


Overview 

The  usage  of  the  four  media  channels  and  their  effects  on  learning  outcomes 
were  examined  and  the  data  collected  was  analyzed  using  hierarchical  regression,  partial 
correlations,  and  linear  regression  and  are  discussed  here.  The  media  channels  were 
instructional  methods  a  user  could  choose  to  engage  in  to  reinforce  the  understanding  of 
course  objectives.  Each  channel  was  used  independent  of  the  other;  however,  some  may 
have  contained  an  element  of  the  other.  For  example,  video  and  avatar  also  include  audio 
channels,  however  those  elements  were  considered  inherent  to  that  particular 
instructional  method. 

Correlations 

Inspection  of  the  correlation  results  in  Table  1  shows  audio  with  a  strong  relation 
with  usage  and  learning  engagement  (Pearson  Correlation  =  .234,  p  <  .05),  and  avatars 
having  the  smallest  correlation  with  learning  engagement.  Learning  engagement 
exhibited  a  highly  significant  correlation  with  affective  reactions  (Pearson  Correlation  = 
.381,  p  <  .01)  and  a  low  negative  correlation  with  cognitive  reactions  (Pearson 
Correlation  =  -  .074,  p  <  .01).  Affective  reactions  were  highly  correlated  with  learning 
(Pearson  Correlation  =  .681  ,P<  .01).  Similarly,  cognitive  reactions  were  highly 
correlated  with  learning  (Pearson  Correlation  =  .3 15,  p  <  .01). 
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Table  1.  Correlations  among  predictors  and  criterion  measures 


Mean 

SD 

1 

2 

3 

4 

5 

6 

7 

8 

1 .  Avatar 
usage 

2.23 

1.36 

a. 

2.  Video 
usage 

2.41 

1.40 

.678** 

a. 

3.  Audio 
usage 

2.16 

1.34 

.422** 

.594** 

a. 

4.  Text 
usage 

3.42 

1.53 

.166 

.240* 

.242* 

a. 

5.  Learning 
engagement 

3.23 

1.00 

.074 

.145 

.234* 

.179 

.836 

6.  Affective 
reactions 

3.27 

.77 

.330** 

.384** 

.329** 

.235* 

.380** 

.808 

7.  Cognitive 
reactions 

3.96 

.81 

.224* 

.196 

-.062 

.074 

.474 

.636 

.774 

8.  Learning 

3.51 

1.05 

.356** 

.329** 

.376“ 

.127 

.204* 

.681** 

.003 

.95 

* p  <  .05  (2 -tailed);  **p  <  .01  (2-tailed);  Cronbach’s  alpha  on  diagonal;  a.  single  item  measure 


Hypotheses  Tests 

Hypotheses  la-d  were  tested  using  the  hierarchical  regression.  Analyses  results 
are  shown  in  Table  2.  HI  a:  Text  presentations  will  have  a  positive  relation  with  learning 
engagement.  Results  for  text  exhibited  a  significant  positive  relationship  with  learning 
engagement  (fi=  A19,p  <  .05).  As  a  result,  Hla  is  supported. 

Hl(b):  Audio  presentation  will  be  more  engaging  than  text  and  account  for 
variance  in  learning  engagement  beyond  that  accounted  for  by  text  presentation  alone. 
Results  in  the  second  model  show  that  audio  accounted  for  significant  incremental 
variance  in  learning  engagement  beyond  that  with  text  alone  (A R  =  .039).  As  a  result, 
Hlb  is  supported. 

Hl(c):  Video  presentations  will  be  more  engaging  than  both  text  and  audio  and 
account  for  variance  in  learning  engagement  beyond  that  accounted  for  by  text  and  audio 
presentation.  Results  for  the  third  variable,  video,  showed  no  significant  increase  in 
incremental  variance  (A R  =  .000).  As  a  result,  Hie  is  not  supported. 
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Hl(d):  Avatar  presentations  will  be  more  engaging  than  text,  audio,  and  video 
and  will  account  for  variance  in  learning  engagement  beyond  that  accounted  for  by  text, 
audio,  and  video  presentations  together.  Finally,  avatars  in  the  fourth  model  also 
showed  no  significant  increase  in  variance  explained  (A R  =  .002).  As  a  result.  Hid  is 
not  supported.  Overall,  the  first  two  models  confirmed  hypotheses  la  &  b  that  address 
the  effects  of  text  and  audio,  but  those  with  Hie  and  d  which  addressed  video  and  avatar 
were  not  supported. 


Table  2.  Results  of  Hierarchical  Regression  Analyses 


Model  1 
(text) 

Model  2 
(text/audio) 

Model  3 

(text/ audio/video ) 

Model  4 

(text/audio/video/avatar) 

Hierarchical 

step 

Ind.  var. 

P 

P 

P 

P 

1. 

Text 

.179* 

.130 

.131 

.131 

2. 

Audio 

.203* 

.209 

.210 

3. 

Video 

-.011 

.025 

4. 

Avatar 

-.053 

F 

3.08 

3.50 

2.311 

1.755 

AR2 

.032* 

.039* 

.000 

.002 

Note:  *p  <  .05.  Dependent  variable:  Learning  Engagement 


Hypotheses  H2a  and  H2b  were  tested  and  analyzed  using  a  partial  correlation 
method  by  controlling  for  the  effects  of  individual  variables.  The  results  pertaining  to  the 
learning  engagement-reactions  criterion  are  presented  here.  Hypotheses  2a:  Higher 
levels  of  learning  engagement  will  lead  to  higher  levels  of favorable  affective  reactions, 
was  supported  as  results  from  the  model  indicated  learning  engagement  was 
significantly  related  to  affective  reactions  while  controlling  for  the  affects  of  cognitive 
reactions  (r  =  .414  ,p  <  .05).  The  results  for  Hypothesis  2b:  Higher  levels  of  learning 
engagement  will  lead  to  a  negative  relation  with  cognitive  reactions  showed  partial 
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support  as  learning  engagement  was  negatively  related  to  cognitive  reactions  while 
controlling  for  the  affects  of  affective  reactions  (r  =  -.185,  p  =  .05). 

Hypothesis  3a  and  3b  were  tested  using  simple  linear  regression  and  the  results 
are  presented  here.  H3a:  Higher  levels  of  affective  reactions  will  lead  to  higher  levels  of 
learning,  was  fully  supported  (/?=  .644 ,p  <  .005).  Hypothesis  3b:  Higher  levels  of 
cognitive  reactions  will  lead  to  higher  levels  of  learning,  was  supported  (J3=  .121,  p  < 
.05).  Results  are  shown  in  Table  3. 


Table  3. 

H3a-b  regression  results 

P 

Affective 

Reactions 

.644* 

Cognitive 

Reactions 

.121** 

F 

41.91 

R2 

.447 

Dependent  variable:  Learning 
*p  < .005 
**p  < .05 
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V.  Discussion 


Summary 

E-leaming  solutions  continue  to  grow  as  organizations  seek  ways  to  educate  and 
train  their  people  most  effectively  and  efficiently.  It  has  been  shown  that  learning 
engagement  enables  other  factors  beneficial  to  favorable  learning  outcomes  (Carini  et  ah, 
2006),  such  as  Kirkpatrick’s  level  one  and  two.  This  study  sought  to  determine  the 
various  levels  of  learning  engagement  induced  by  text,  audio,  video,  and  avatar 
instructional  methods  and  how  that  related  to  learning  outcomes. 

This  issue  is  particularly  important  in  the  case  of  understanding  e-leaming 
effectiveness  due  to  the  influx  of  technology-based  education  and  training  solutions  being 
offered  today  without  empirical  evidence  of  favorable  outcomes.  Applying  the  emotional 
interest  theory  as  a  basis  for  understanding  initial  learning  engagement  in  e-leaming 
environments,  this  study  was  particularly  interested  in  the  marketing  and  usage  of 
innovative  avatar  instmctional  methods  and  other  media  channels. 

Using  emotional  appeal  through  its  high-resolution  graphical  interface, 
interactivity,  and  other  factors,  this  technology  has  become  widely  adopted  and  believed 
to  be  an  effective  method  of  instmction.  However,  there  have  been  no  known  research 
studies  to  provide  definitive  results  linking  avatars  and  other  media  channels  to  e-learning 
effectiveness.  Unfortunately,  the  emotional  appeal  of  a  multitude  of  e-learning 
technological  innovations  has  seemingly  turned  people’s  focus  away  from  cognitive  and 
learning  science  principles,  indicating  more  attention  needs  to  focus  in  this  area  of 
research  to  help  ensure  education  investments  are  leading  to  desired  learning  outcomes. 
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Theoretical  Foundations 


In  order  to  link  media  usage  to  learning  engagement,  this  study  used  the  media 
richness  theory,  media  synchronicity  theory,  and  the  emotional  interest  theory  to  predict 
the  learning  engagement  levels  of  the  four  media  channels  used  in  the  e-learning  program 
examined,  thereby  allowing  for  predicted  learning  outcomes.  It  was  thought,  that  due  to 
its  learning  capacity  (richness),  multiple  cues  (vocal  inflections,  gestures),  dual-modality, 
low-speed  conveyance  of  new  information,  and  quick  feedback  (convergence),  that 
avatars  would  be  the  most  effective  at  generating  learning  engagement. 

Furthennore,  it  was  predicted  that  higher  levels  of  learning  engagement  would 
lead  to  various  levels  of  user  reactions  (Kirkpatrick  level  1):  positively-related  affective 
reactions  and  negatively-related  cognitive  reactions.  It  is  intuitive  to  think  that  someone 
actively  engaged  in  an  optional  learning  activity  is  likely  to  be  enjoying  the  experience 
(affective  reaction),  or  they  would  not  have  escalated  their  level  of  activity.  Conversely, 
the  negative  relation  predicted  with  cognitive  reactions  was  detennined  through  a 
consideration  of  the  cognitive  dissonance  theory,  which  posits  that  as  people  learn,  they 
are  entering  an  unknown  and  sometimes  uncomfortable  experience  and  would  thus  reflect 
negatively  on  learner’s  cognitive  reactions. 

Finally,  Kirkpatrick’s  Level  2  (learning)  was  used  to  link  Level  1  to  learning 
effectiveness  by  predicting  higher  levels  of  both  affective  and  cognitive  reactions  would 
lead  to  higher  levels  of  understanding.  Using  Alavi  and  Leidner’s  concept  of 
psychological  learning  processes  that  lead  to  learning  outcomes,  the  paradox  of  negative 
cognitive  reactions  is  explained:  as  a  learner  successfully  progresses  through  the  process 
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of  dissonance  (learning  engagement  — >  cognitive  dissonance  — >  sensmaking  — >  positive 
cognitive  reactions),  the  result  would  be  higher  levels  of  understanding. 

Findings 

First,  neither  avatars  nor  video  were  found  to  be  significantly  related  to  learning 
engagement.  The  hierarchical  regression  analysis  revealed  the  predicted  incremental 
validity  of  the  media  channels  did  not  exist.  Conversely,  the  two  channels  thought  to  be 
least  engaging,  text  and  audio,  revealed  strong  correlations  with  learning  engagement. 

These  findings  could  possibly  be  related  an  access  factor:  the  avatar  and  video 
channels  are  restricted  to  the  internet-connected  computer  being  used,  whereas  the  audio 
and  text  provides  the  learner  freedom  of  movement  by  having  the  ability  to  download 
and/or  print  the  material  and  use  it  whenever/wherever  they  choose  to  engage  in  learning. 

Another  factor  may  be  the  demographics  of  the  participants  and  their  typical  style 
of  learning:  educated  Air  Force  captains/equivalents  with  5-7  years  of  service  have  been 
through  years  of  education  and  training  and  may  view  avatar  “vignettes”  or  video  clips  as 
a  waste  of  their  time  and  prefer  to  simply  engage  in  reading  text  for  example. 

A  second  key  finding  was  that,  while  controlling  for  cognitive  reactions,  there 
was  a  significant  partial  correlation  between  learning  engagement  and  affective  reactions. 
Furthennore,  the  negative  relation  with  learning  engagement  and  cognitive  reactions  was 
found  to  be  partially  supported  while  controlling  for  affective  reactions. 

Finally,  as  expected,  a  regression  analysis  showed  higher  levels  of  affective 
reactions  were  highly  related  to  subjective  learning.  The  finding  for  higher  levels  of 
cognitive  reactions  leading  to  understanding  was  partially  supported.  These  findings 
confirm  the  belief  that  a  learner  who  is  enjoying  the  learning  experience  (affective)  will 
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be  more  attentive  and  receptive  to  learning  objectives,  resulting  in  increased 
understanding.  The  finding  for  cognitive  reactions  leading  to  understanding  may  have 
only  been  partially  supported  due  to  a  continuation  of  the  cognitive  dissonance  theory. 
Although  the  learner  has  progressed  favorably  through  the  psychological  learning 
process,  there  is  still  a  sense  of  imbalance,  or  a  heightened  awareness  of  how  much  they 
still  don’t  know. 

limitations 

One  possible  limitation  of  the  study  pertains  to  the  avatar  technology  used. 
Although  defined  multiple  ways,  avatars  have  come  to  be  known  as  something 
controllable,  interactive,  and  composed  of  immersive  engagement,  and  the  instructional 
method  may  be  much  different  from  what  some  users  expected.  Although  considered  the 
“richest”  media  in  the  study,  and  have  the  most  appropriate  levels  of  synchronicity  for 
learning,  the  vicarious  method  of  learning  through  vignettes  may  limit  the  learner’s 
expected  sense  of  engagement. 

Furthennore,  a  consideration  of  participant’s  education  level  (bachelor’s  degree  or 
higher)  may  indicate  that  an  avatar  or  video  channel  for  learning  simply  may  not  be 
cognitively  appealing.  By  the  time  a  participant  has  finished  the  SOS  DL  program,  they 
have  been  in  the  Air  Force  around  5-7  years.  After  years  of  education  and  training  on  a 
number  of  issues,  anything  other  than  simple  text  or  audio  delivery  methods  may  be 
perceived  as  something  that  slows  them  down,  causing  them  to  disregard  the  media 
channels  altogether,  or  not  feel  satisfactorily  engaged  with  the  technique. 

Another  potential  limitation  of  this  study  pertains  to  sample  size.  Although 
eleven  thousand  students  were  enrolled  in  the  course,  the  research  timeline  permitted 
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only  two  months  of  data  collection  and  resulted  a  sample  of  95.  Considering  participants 
course  completion  timeframe  of  18-months,  more  time  for  data  collection  would  have 
increased  the  sample  size  and  perhaps  altered  the  results  significantly. 

A  fourth  limitation  may  be  the  layout  of  the  course  interface  that  may  discourage 
the  use  of  available  media  channels.  Several  of  the  media  features  were  difficult  to  locate 
and  might  result  in  users  not  being  aware  of  the  option,  or  view  them  as  insignificant  due 
to  their  subdued  existence.  The  avatar  exercises,  for  example,  are  well  “below  the  fold” 
at  the  bottom  of  learning  modules  without  being  “advertised”  sufficiently  anywhere  else 
in  the  program.  Both  the  audio  and  video  “links”  were  difficult  to  locate  and  did  not 
work  at  times.  These  potential  limiting  factors  prohibit  maximum  media  channel  usage 
that  may  have  otherwise  returned  different  results. 

The  scope  of  this  research  may  also  be  a  limiting  factor  since  the  data  collected 
was  limited  to  Level  1  and  2  learning  outcomes.  The  consideration  of  any  post-training 
behavioral  changes  and/or  other  desired  organizational  outcomes  resulting  from  the 
Squadron  Officer  School  Distance  Learning  training  experience  was  outside  the  scope  of 
this  research. 

Finally,  because  data  was  collected  from  a  single-item  survey,  common  method 
bias  (CMB)  may  have  influenced  the  research  results  of  this  study.  CMB  has  been 
identified  as  a  main  source  of  measurement  error  and  problematic  when  using  single-item 
measuring  techniques  (Podsakoff,  2003).  It  is  possible  that  different  results  would  have 
been  obtained  if  a  variety  of  measurement  items  had  been  used. 
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Theoretical  Implications 

This  study  concentrated  on  optional  media  usage  and  its  initiation  of  learning 
engagement  and  the  subsequent  linkages  to  e-learning  effectiveness.  In  doing  so,  this 
research  has  exploited  two  main  theoretical  themes:  educational  psychology  and 
information  technology,  and  hypothesized  avatars  to  be  the  most  engaging  media 
channel. 

Educational  psychology  addressed  the  psychological  learning  process  and  how  it 
leads  to  learning  outcomes.  Infonnation  technology  theory  addressed  the  most  effective 
ways  to  communicate  in  order  to  optimize  learning  effectiveness.  In  relation  to  these, 
some  perspectives  have  been  discussed  and  the  results  of  their  application  in  this  study 
have  been  shown.  This  section  will  summarize  the  theoretical  concepts  used  to  develop 
the  hypotheses  of  this  study,  and  discuss  how  they  relate  to  the  findings. 

Information  Technology,  Learning  Engagement,  and  Educational  Psychology 

Figure  1  introduced  the  integration  of  an  instructional  strategy  with  information 
technology  in  order  to  influence  the  psychological  learning  process.  This  study  focused 
on  the  information  technology  element  of  instructional  system  designs  by  examining  the 
use  of  media  channels  to  influence  learning  effectiveness.  It  was  hypothesized  that  media 
channels  would  induce  varying  levels  of  learning  engagement  as  an  element  of  the 
psychological  learning  process. 

Based  on  the  emotional  interest  theory,  it  was  thought  new  and  innovative  avatar 
technology  would  generate  the  greatest  interest,  thereby  inducing  user’s  emotional 
engagement  to  a  greater  degree  than  the  other  channels.  Consequently,  as  the  learners  in 
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this  study  indicated  the  least  engagement  with  avatars,  it  is  a  noteworthy  consideration 
that  avatars  are  not  as  cognitively  stimulating  as  first  thought. 

Newman  described  learning  engagement  as  “the  student’s  psychological 
investment  in  learning”  (1989).  This  study  examined  two  seminal  information 
technology  theories  that  relate  to  affecting  the  psychological  learning  process  to  help 
predict  which  media  channel  would  be  the  most  engaging  and  lead  to  higher  levels  of 
learning  effectiveness.  The  media  richness  theory  was  used  to  identify  avatars  as  the 
most  rich,  and  therefore,  most  engaging  and  possess  the  greatest  “learning  capacity”  that 
would  lead  to  the  optimal  learning  effectiveness.  Avatars  were  chosen  mainly  because  of 
two  key  media  richness  theory  qualities  that  gave  them  an  advantage  over  the  other 
channels:  1)  multiple  cues  (vocal  variety  and  gestures),  and  2)  immediate  feedback 
(question/answer  feature).  Next,  media  synchronicity  theory  was  used  to  identify  avatars 
as  having  optimal  conveyance  (information  transmission)  and  convergence  (reaching  a 
mutual  understanding)  qualities  of  all  media  channels  used  in  the  study. 

The  research  results  presented  in  this  study  indicate  the  avatar  technology  was  in 
fact  the  least  engaging  of  the  four  media  channels.  The  theoretical  implication  of  this 
result  is  congruent  with  initial  reason  for  this  study:  media  affects  on  e-learning  are  not 
sufficiently  understood  and  studies  like  the  one  presented  here  are  needed  to  clarify  any 
common  assumptions  about  such  things  as  emotional  interest  and  learning  engagement. 

For  example,  although  emotional  interest  has  been  shown  to  engage  learners 
initially,  multimedia  e-leaming  engagement  may  be  short-lived  once  it  is  apparent  there 
is  little  cognitive  value  contained  therein.  Clark  once  observed  that  e-learning  learning  is 
not  necessarily  more  engaging  and  “can  mislead  students  into  investing  less  mental  effort 
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to  learn”  (2009).  This  is  consistent  with  what  Dennis  describes  as  the  paradox  of 
richness:  a  rich  media  increases  engagement  while  simultaneously  decreasing  the  ability 
to  process  messages  due  to  rich  elements  that  distract  from  relevant  infonnation  (2005). 
This  paradox  may  provide,  at  least  in  part,  a  theoretical  explanation  for  the  unsupported 
results  regarding  avatars  and  learning  engagement. 

Applications 

This  study  hypothesized  about  learning  outcomes  based  on  engagement  elements 
of  the  learning  process,  but  used  a  theoretical  foundation  that  supported  the  ultimate 
objective  of  discovering  the  learning  effectiveness  of  an  e-leaming  program.  The 
research  findings  presented  here  can  be  useful  for  any  organization  considering 
e-leaming  solutions  for  their  education  and  training  requirements  by  placing  the  same 
emphasis  on  proven  pedagogical  methods  and  outcomes  rather  than  on  the  enjoyment 
factor. 

Opportunities  for  Future  Research 

The  present  study  provided  a  foundational  understanding  of  the  effects  of  media 
usage  in  e-learning  and  its  affect  on  learning  effectiveness.  Considering  the 
demographics  of  the  participants  in  this  study,  it  would  be  interesting  to  use  the  theory 
presented  here  to  examine  another  e-leaming  course  with  different  levels  of  education, 
age,  and  experience.  Similar  to  the  SOS  Distance  Learning  course,  the  U.S.  Air  Force 
has  another  Professional  Military  Education  course  designed  for  its  non-commissioned 
officers,  for  example.  It  is  recommended  that  the  generalizable  theory  presented  here  be 
applied  under  such  conditions  in  order  to  discover  how  much  individual  demographic 
differences  affect  learning  engagement  and  learning  outcomes. 
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Another  opportunity  would  be  to  research  media  use  in  e-leaming  from  varying 
generational  scopes.  Discussing  how  to  manage  “Generation  Y,”  Tulgan  says  people 
mistakenly  think  this  generation  wants  to  leam  only  from  computers,  when  in  fact  they 
really  only  want  to  leam  things  that  are  easy  to  leam  from  computers  and  still  value 
human  contact  (2009,  p.  14).  Identifying  generational  differences  in  acceptance  and 
engagement  with  innovative  e-leaming  technologies  such  as  avatars  would  provide 
valuable  insight  into  important  ISD  considerations. 

Thesis  Conclusion 

This  thesis  explored  the  affects  of  media  usage  and  the  linkages  to  e-learning 
outcomes.  The  purpose  of  this  research  was  to  develop  a  generalizable  theory  that  can  be 
used  to  assess  the  learning  effectiveness  of  various  media  types  used  in  e-leaming 
environments.  In  order  to  achieve  this  objective,  an  extensive  literature  review  was 
conducted  involving  educational  psychology,  information  technology,  and  learning 
outcomes.  As  a  result  of  this  literature  review,  a  total  of  eight  testable  hypotheses  were 
proposed.  Using  a  single-item  survey  the  research  methodology  involved  the  collection 
of  quantitative  data  from  the  U.S.  Air  Force  Squadron  Officer  School  Distance  Learning 
course. 

The  results  of  this  thesis  were  detennined  through  a  study  of  eight  statistical 
models.  Overall,  this  research  helps  fill  a  gap  in  existing  e-leaming  effectiveness 
research  by  providing  an  understanding  of  how  media  usage  in  e-learning  can  induce 
learning  engagement  and  affect  learning  outcomes  in  e-leaming  environments. 
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